Field of the Invention
The invention relates generally to musical instruments and, in particular, to techniques suitable for use in portable device hosted implementations of musical instruments for capture and rendering of musical performances.
Description of the Related Art
The field of mobile music has been explored in several developing bodies of research. See generally, G. Wang, Designing Smule's iPhone Ocarina, presented at the 2009 on New Interfaces for Musical Expression, Pittsburgh (June 2009) and published at https://ccrma.stanford.edu/˜ge/publish/ocarina-nime2009.pdf. One application of this research has been the Mobile Phone Orchestra (MoPhO), which was established in 2007 at Stanford University's Center for Computer Research in Music and Acoustics and which performed its debut concert in January 2008. The MoPhO employs more than a dozen players and mobile phones which serve as a compositional and performance platform for an expanding and dedicated repertoire. Although certainly not the first use of mobile phones for artistic expression, the MoPhO has been an interesting technological and artistic testbed for electronic music composition and performance. See generally, G. Wang, G. Essl and H. Penttinen, MoPhO: Do Mobile Phones Dream of Electric Orchestras? in Proceedings of the International Computer Music Conference, Belfast (August 2008).
Mobile phones and other portable computing devices are growing in sheer number and computational power. Hyper-ubiquitous and deeply entrenched in the lifestyles of people around the world, they transcend nearly every cultural and economic barrier. Computationally, the mobile phones and pad-type devices of today offer speed and storage capabilities comparable to desktop computers from less than ten years ago, rendering them surprisingly suitable for real-time sound synthesis and other musical applications. Like traditional acoustic instruments, the mobile phones are intimate sound producing devices. By comparison to most instruments, they are somewhat limited in acoustic bandwidth and power. However, mobile phones have the advantages of ubiquity, strength in numbers, and ultramobility, making it feasible to hold jam sessions, rehearsals, and even performance almost anywhere, anytime.
Research to practically exploit such devices has been ongoing for some time. For example, a touch-screen based interaction paradigm with integrated musical synthesis on a Linux-enabled portable device such as an iPaq™ personal digital assistant (PDA) was described by Geiger. See G. Geiger, PDa: Real Time Signal Processing and Sound Generation on Handheld Devices, in Proceedings of the International Computer Music Conference, Singapore (2003); G. Geiger, Using the Touch Screen as a Controller for Portable Computer Music Instruments in Proceedings of the International Conference on New Interfaces for Musical Expression, Paris (2006). Likewise, an accelerometer based custom-made augmented PDA capable of controlling streaming audio was described by Tanaka. See A. Tanaka, Mobile Music Making, in Proceedings of the 2004 Conference on New Interfaces for Musical Expression, pages 154-156 (2004).
Indeed, use of mobile phones for sound synthesis and live performance was pioneered by Schiemer in his Pocket Gamelan instrument, see generally, G. Schiemer and M. Havryliv, Pocket Gamelan: Tuneable Trajectories for Flying Sources in Mandala 3 and Mandala 4, in Proceedings of the 2006 Conference on New Interfaces for Musical Expression, pages 37-42, Paris, France (2006), and remains a topic of research. The MobileSTK port of Cook and Scavone's Synthesis Toolkit (STK) to Symbian OS, see G. Essl and M. Rohs, Mobile STK for Symbian OS, in Proceedings of the International Computer Music Conference, New Orleans (2006), was perhaps the first full parametric synthesis environment suitable for use on mobile phones. Mobile STK was used in combination with accelerometer and magnetometer data in ShaMus to allow purely on-the-phone performance without any laptop. See G. Essl and M. Rohs, ShaMus—A Sensor-Based Integrated Mobile Phone Instrument, in Proceedings of the International Computer Music Conference, Copenhagen (2007).
As researchers seek to transition their innovations to commercial applications deployable to modern handheld devices such as iPad™ and iPhone® mobile digital devices (available from Apple Inc.) and other platforms operable within the real-world constraints imposed by processor, memory and other limited computational resources thereof and/or within communications bandwidth and transmission latency constraints typical of wireless networks, practical challenges present.
Improved techniques and solutions are desired.